Papers by Mohammadreza Tayaranian Hosseini
Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation (2023.findings-eacl)
Copied to clipboard
Mohammadreza Tayaranian Hosseini, Alireza Ghaffari, Marzieh S. Tahaei, Mehdi Rezagholizadeh, Masoud Asgharian, Vahid Partovi Nia
| Challenge: | Previously, researchers focused on lower bit-width integer data types for forward propagation and backward propagation of language models to save memory and computation. |
| Approach: | They use integer arithmetic for both forward and back propagation in the fine-tuning of BERT. |
| Outcome: | The proposed method improves on the GLUE and SQUAD benchmarks. |